AUTHORS: M. Madhiarasan, S. N. Deepa

Download as PDF

ABSTRACT: This paper proposes a novel method to select hidden neurons in ELMAN neural networks for wind speed prediction application. Either over fitting or under fitting problem caused due to the random choice of hidden neuron numbers in artificial neural network. This paper suggests the solution to solve either over fitting or under fitting problems. In order to select proper hidden neuron numbers, 75 different criteria tested by the means of statistical errors. The simulation results proved that proposed approach improves the accuracy and reduce the error to the least. The perfect building of ELMAN network with five inputs using fixation criteria is validated based on convergence theorem. To evaluate the performance of the proposed approach simulation were performed on real time wind data. Comparative analysis has performed to select the hidden neuron numbers in neural networks. The presented approach is very simple, with the least error, and more effective to select the amount of hidden neurons in ELMAN neural network.

KEYWORDS: Novel Criteria; Hidden Neurons; ELMAN Neural Network; Prediction; Wind Speed.

REFERENCES:

[1] S. N. Sivanandam, S. Sumathi, S. N. Deepa, Introduction to Neural Networks using Matlab 6.0, Tata McGraw Hill, 2008.

[2] Gaurang Panchal, Amit Ganatra, Y. P. Kosta and Devyani Panchal, Behaviour analysis of multilayer perceptrons with multiple hidden neurons and hidden layers, International Journal of Computer Theory and Engineering, Vol.3, No.2, 2011, pp. 332-337.

[3] David Hunter, Hao Yu, Michael. S. Pukish III, Janusz Kolbusz, Bogdon. M. Wilamowski, Selection of proper neural network sizes and architecture- A comparative study, IEEE Transactions on Industrial Informatics, Vol.8, No.2, 2012, pp. 228-240.

[4] Shuxiang Xu, Ling Chen, A novel approach for determining the optimal number of hidden layer neurons for FNN’s and its application in data mining, 5th International Conference on Information Technology and Application (ICITA), 2008, pp. 683-686.

[5] Jinchuan Ke, Xinzhe Liu, Empirical analysis of optimal hidden neurons in neural network modeling for stock prediction, Pacific, Asia workshop on Computational Intelligence and Industrial Application, Vol.2, 2008, pp. 828- 832.

[6] Saurabh Karsoliya, Approximating number of hidden layer neuron in multiple hidden layer BPNN architecture, International Journal of Engineering Trends and Technology, Vol.31, No.6, 2012, pp. 714-717.

[7] Shih-Chi Huang, Yih-Fang Huang, Bounds on the number of hidden neurons in multilayer perceptrons, IEEE Transaction on Neural Network, Vol.2, No.1, 1991, pp. 47- 55.

[8] Arai. M, Bounds on the number of hidden units in binary-valued three-layer neural networks, Neural Networks, Vol.6, 1993, pp. 855-860.

[9] Masafumi Hagiwara, A simple and effective method for removal of hidden units and weights, Neuro Computing, Vol. 6, No.2, 1994, pp. 207-218.

[10] Noboru Murata, Shuji Yoshizawa, Shun-Ichi Amari, Network Information Criterion determining the number of hidden units for an artificial neural network model, IEEE Transaction on Neural Networks, Vol.5, No.6, 1994, pp. 865-872.

[11] Jin-Yan Li, Chow. T. W. S, Ying-Lin Yu, The estimation theory and optimization algorithm for the number of hidden units in the higherorder feed forward neural network, Proceeding IEEE International Conference on Neural Networks, Vol.3, 1995, pp. 1229-1233.

[12] Onoda. T, Neural network information criterion for the optimal number of hidden units, Proceeding IEEE International Conference on neural networks, Vol.1, 1995, pp. 275-280.

[13] Tamura. S, Tateishi. M, Capabilities of a fourlayered feed forward neural network: four layer versus three, IEEE Transaction on neural network, Vol. 8, No.2, 1997, pp. 251-255.

[14] Osamu Fujita, Statistical estimation of the number of hidden units for feed forward neural network, Neural Network, Vol.11, 1998, pp. 851-859.

[15] Keeni. K, Nakayama. K and Shimodaira. H, Estimation of initial weights and hidden units for fast learning of multilayer neural networks for pattern classification, International Joint Conference on Neural Networks, Vol.3, 1999, pp. 1652-1656.

[16] Guang-Bin Huang, Learning capability and storage capacity of two-hidden layer feed forward networks, IEEE Transactions on Neural Networks, Vol.14, No.2, 2003, pp. 274- 281.

[17] H. C. Yuan, F. L. Xiong and X. Y. Huai, A method for estimating the number of hidden neurons in feed-forward neural networks based on information entropy, Computers and Electronics in Agriculture, Vol.40, 2003, pp. 57-64.

[18] Zhaozhi Zhang, Xiaomin Ma, Yixian Yang, Bounds on the number of hidden neurons in three-layer binary neural networks, Neural Network, Vol.16, 2003, pp. 995-1002.

[19] K. Z. Mao, Guang-Bin Huang, Neuron selection for RBF neural network classifier based on data structure preserving criterion, IEEE Transactions on Neural Networks, Vol.16, No.6, 2005, pp. 1531-1540.

[20] E. J. Teoh, K. C. Tan, C. Xiang, Estimating the number of hidden neurons in a feed forward network using the singular value decomposition, IEEE Transactions on Neural Networks, Vol.17, No.6, 2006, pp.1623-1629.

[21] Xiaoqin Zeng, Daniel S. Yeung, Hidden neuron pruning of multilayer perceptrons using a quantified sensitivity measure, Neuro Computing, Vol.69, 2006, pp. 825-837.

[22] Bumghi Choi, Ju-Hong Lee, Deok-Hwan Kim, Solving local minima problem with large number of hidden nodes on two layered feed forward artificial neural networks, Neuro Computing, Vol.71, No. 16-18, 2008, pp. 3640-3643.

[23] Min Han, Jia Yin, The hidden neurons selection of the wavelet networks using support vector machines and ridge regression, Neuro Computing, Vol.72, No.1-3, 2008, pp. 471- 479.

[24] Nan Jiang, Zhaozhi Zhang, Xiaomin Ma, Jian Wang, The lower bound on the number of hidden neurons in multi-valued multi threshold neural networks, Second International Symposium on Intelligent Information Technology Application, Vol.1, 2008, pp.103- 107.

[25] Stephen Trenn, Multilayer perceptrons: Approximation order and necessary number of hidden units, IEEE Transactions on Neural Networks, Vol. 19, No.5, 2008, pp. 836-844.

[26] Katsunari Shibata, Yusuke Ikeda, Effect of number of hidden neurons on learning in largescale layered neural networks, ICROS-SICE International Joint Conference, 2009, pp. 5008-5013.

[27] Doukin. C. A,Dargham. J. A, Chekima. A, Finding the number of hidden neurons for an MLP neural network using coarse to fine search technique. 10th International Conference on Information Sciences Signal Processing and their Applications (ISSPA), 2010, pp. 606-609.

[28] Junfang Li, Buhan Zhang, Chengxiong Mao, Guang Long Xie, Yan Li, Jiming Lu, Wind speed prediction based on the Elman recursion neural networks, International Conference on Modelling, Identiication and Control, 2010, pp. 728-732.

[29] Jianye Sun, Learning algorithm and hidden node selection scheme for local coupled feed forward neural network classifier, Neuro Computing, Vol.79, 2012, pp. 158-163.

[30] Ramadevi. R, Sheela Rani. B, Prakash. V, Role of hidden neurons in an Elman recurrent neural network in classification of cavitation signals, International Journal of Computer Application, Vol.37, No.7, 2012, pp. 9-13.

[31] K. Gnana Sheela, S. N. Deepa, Review on methods to fix number of hidden neurons in neural networks, Mathematical Problems in Engineering, Vol.2013, 2013, pp.1-11.

[32] Guo Qian, Hao Yong, Forecasting the rural per capita living consumption based on Matlab BP neural network, International Journal of Business and Social Science, Vol.4, No.17, 2013, pp.131-137.

[33] KuldipVora, ShrutiYagnik, A new technique to solve local minima problem with large number of hidden nodes on feed forward neural network, International Journal of Engineering Development and Research, Vol.2, No.2, 2014, pp.1978-1981.

[34] Siddhaling Urolagin, K. V. Prema, N. V. Subba Reddy, Generalization capability of artificial neural network incorporated with pruning method, Lecture Notes in Computer Science, Vol.7135, 2012, pp. 171-178.

[35] Jeffrey L. Elman, Finding structure in time. Cognitive Science, Vol.14, 1990, pp. 179-211.

[36] Lin. F. J, Hung. Y. C, FPGA-based Elman neural network control system for linear ultrasonic motor, IEEE Trans ultras on Ferro electr Freq Control, Vol.56, No.1, 2009, pp. 101-113.

[37] Liu Hongmei, Wang Shaoping, Ouyang Pingchao, Fault diagnosis based on improved Elman neural network for a hydraulic servo system, Proceeding International Conference on Robotics, Automation Mechatronics, 2006, pp. 1-6.

[38] Xiang Li, Guanrong Chen, Zengqiang Chen, Zhuzhi Yuan, Chaotifying linear Elman networks, IEEE Transaction neural network, Vol.13, No.5, 2002, pp. 1193-1199.

[39] Morris. A. J, Zhang. J, A sequential learning approach for single hidden layer neural network, Neural Networks, Vol.11, No.1, 1998, pp. 65-80.

[40] H. K. Dass, Advanced Engineering Mathematics, S. CHAND & Company Ltd, First edition 1988, reprint 2009.